Combining Forecasts: The End of the Beginning or the Beginning of the End?

Research from over 200 studies demonstrates that combining forecasts produces consistent but modest gains in accuracy. However, this research does not define well the conditions under which combining is most effective nor how methods should be combined in each situation. Rule-based forecasting can be used to define these conditions and to specify more effective combinations. Introduction “Combining Forecasts” by Clemen (1989) is a milestone on the topic of combining forecasts. His paper summarizes a great amount of research from different fields in a concise and readable manner. It will save time for those who do further research on combined forecasts. This commentary examines what can be concluded from the research to date, with an emphasis on what has not been learned. I make suggestions about alternative research strategies: ‘Realistic simulation’ and ‘rule-based forecasting’ are especially promising. Results to Date: The End of the Beginning As noted by Clemen, past research has produced two primary conclusions, one expected and one surprising. The expected conclusion is that combined forecasts reduce error (in comparison with the average error of the component forecasts). The unexpected conclusion is that the simple average performs as well as more sophisticated statistical approaches. These conclusions are summarized succinctly by Clemen when he suggests that the forecasting practitioner should “combine and average.” Beyond this conclusion, however, the research has produced few guidelines that are specific enough to be used by forecasters. Research Strategies: The Beginning of the End The 209 studies reviewed by Clemen yielded surprisingly little in the way of guidelines for forecasters. Perhaps there has been too much of a reliance on classical statistical procedures. Additional studies of the type already published are unlikely to produce significant gains. In this respect, we are at the beginning of the end. Alternative strategies are considered here. Among these are (1) meta-analysis, (2) realistic simulations, and (3) rule-based forecasting. * Support for this paper was provided by the U.S. Navy Personnel Research and Development Center, San Diego, CA 92152, USA.